System and method for candidate generation and new features designed for the detection of flat growths

ABSTRACT

A method for generating candidates from a digital image includes considering at least one point x that may lie on a polypoid structure, determining whether the point x satisfies a first predetermined set of conditions, for each point x that satisfies the predetermined set of conditions, identifying each neighbor point y within a predetermined distance of point x that satisfies a second predetermined set of conditions, determining a gradient vector v 1  for point x and identifying a first half-line to which the gradient vector v 1  belongs, determining a gradient vector v 2  for point y and identifying a second half-line to which the gradient vector v 2  belongs, calculating an intersection score that represents how close the first and second half-lines come to intersecting, and identifying point x as a candidate when a candidate score is greater than a predetermined value, wherein the candidate score is the sum of intersection scores for all neighbor points y.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims the benefit of U.S. Provisional PatentApplication No. 61/176,984, filed May 11, 2009, entitled “System AndMethod For Candidate Generation And New Features Designed For TheDetection Of Flat Polyps,” the contents of which are herein incorporatedby reference in their entirety.

TECHNICAL FIELD

The present disclosure generally relates to medical image processing andmore particularly to the detection of candidates from digital medicalimages.

BACKGROUND

The field of medical imaging has seen significant advances since thetime X-Rays were first used to determine anatomical abnormalities.Medical imaging hardware has progressed in the form of newer machinessuch as Medical Resonance Imaging (MRI) scanners, Computed AxialTomography (CAT) scanners, etc. Because of large amount of image datagenerated by such modern medical scanners, there has been and remains aneed for developing image processing techniques that can automate someor all of the processes to determine the presence of anatomicalabnormalities in scanned medical images.

Recognizing anatomical structures within digitized medical imagespresents multiple challenges. For example, a first concern relates tothe accuracy of recognition of anatomical structures within an image. Asecond area of concern is the speed of recognition. Because medicalimages are an aid for a doctor to diagnose a disease or condition, thespeed with which an image can be processed and structures within thatimage recognized can be of the utmost importance to the doctor reachingan early diagnosis. Hence, there is a need for improving recognitiontechniques that provide accurate and fast recognition of anatomicalstructures and possible abnormalities in medical images.

Digital medical images are constructed using raw image data obtainedfrom a scanner, for example, a CAT scanner, MRI, etc. Digital medicalimages are typically either a two-dimensional (“2-D”) image made ofpixel elements or a three-dimensional (“3-D”) image made of volumeelements (“voxels”). Such 2-D or 3-D images are processed using medicalimage recognition techniques to determine the presence of anatomicalstructures such as cysts, tumors, polyps, etc. Given the amount of imagedata generated by any given image scan; it is preferable that anautomatic technique should point out anatomical features in the selectedregions of an image to a doctor for further diagnosis of any disease orcondition.

One general method of automatic image processing employs feature basedrecognition techniques to determine the presence of anatomicalstructures in medical images. However, feature based recognitiontechniques can suffer from accuracy problems.

Automatic image processing and recognition of structures within amedical image is generally referred to as Computer-Aided Detection(CAD). A CAD system can process medical images and identify anatomicalstructures including possible abnormalities for further review. Suchpossible abnormalities are often called candidates and are considered tobe generated by the CAD system based upon the medical images. One typeof CAD candidate generator relies upon the cutting plane images providedby MRI and CT scanners. Another is a divergent gradient field response(DGFR) candidate generator (CG) and is considered a good replacement forthe cutting planes candidate generator for the detection of possibleabnormalities. The DGFR candidate generator is considered an improvedtechnique over the cutting planes candidate generator because itovercomes the main limitation of the cutting planes CG, namely, its 2Dlimitations. The main idea in DGFR is to detect areas where the(inverted) gradient of the image correlates well with a 3D divergentgradient field, such as the one obtained when considering the (inverted)gradient of 3D Gaussian function. The same principle applies to“convergent” gradient fields and the various concepts can be illustratedusing either convergent or divergent fields interchangeably.

This model of a convergent gradient field is well adapted to findingpossible anatomical abnormalities such as, for example, cysts, tumors,and polyps from digital medical images. For example, the DGFR candidategenerator is well adapted to identifying possible colon polyps fromcomputed tomography (CT) images, since most of the polyps have roughlythe shape of a hemisphere attached to the colon wall. The image gradientfield for a hemispherical growth is roughly convergent towards the coreof the polyp. FIG. 1 illustrates an exemplary hemispherical polypattached to a colon wall with gradient field vectors pointing toward thecore of the polyp. The approach of the DGFR technique is to perform avector correlation of the image gradient with the gradient of aGaussian. If the variance σ of the Gaussian is well adapted to the sizeof the polyp, this correlation will be high at the core or center of thepolyp. In order to handle polyps of various sizes, the correlation withthe divergent vector field is performed at multiple values of σ, and theresponses at the various scales are then combined to produce candidates,usually by taking the location of the maximum response across scales andthen applying a threshold to this “combined” response image. FIG. 2illustrates the correlation of the image gradient of an exemplary polypwith a gradient of a Gaussian of standard deviation σ.

However, such conventional image processing and analysis systems andtechniques as described above still often miss potential candidateswithin a digital medical image. This is due in large part to the factthat such anatomical abnormalities are often not as close to perfectlyhemi-spherical as the above DGFR system assumes.

Therefore, a need exists for an improved system and method for detectingand generating candidates from a digital medical image that can betterdetect asymmetric, flat, and otherwise difficult to detect anatomicstructures.

SUMMARY OF THE INVENTION

According to an aspect of the invention, a method is provided forgenerating candidates from a digital image, the method includingconsidering at least one point x that may lie on a polypoid structure,and determining whether the point x satisfies a first predetermined setof conditions. For each point x that satisfies the predetermined set ofconditions, the method includes identifying each neighbor point y withina predetermined distance of point x that satisfies a secondpredetermined set of conditions, determining a gradient vector v₁ forpoint x and identifying a first half-line to which the gradient vectorv₁ belongs, determining a gradient vector v₂ for point y and identifyinga second half-line to which the gradient vector v₂ belongs, calculatingan intersection score that represents how close the first and secondhalf-lines come to intersecting, and identifying point x as a candidatewhen a candidate score is greater than a predetermined value. Thecandidate score is the sum of intersection scores for all points y. Inaccordance with an aspect of the invention, the intersection score isdefined by

${{- \frac{v_{1} \times \left( {y - x} \right)}{{v_{1} \times \left( {y - x} \right)}}} \cdot \frac{v_{1} \times v_{2}}{{v_{1} \times v_{2}}}},$where v₁ is the gradient vector at point x and v₂ is the gradient vectorat point y. Determining whether a point x satisfies the firstpredetermined set of conditions can include determining that the point xis located on the air-tissue interface, and determining that the point xhas a gradient magnitude greater than a predetermined gradient value.According to a further aspect of the invention, determining whether apoint y satisfies the second predetermined conditions can includedetermining that the point y is located on the air-tissue interface,determining that the point y has a gradient magnitude greater than apredetermined gradient value, determining that the point y is locatedwithin a predetermined distance range of the point x, determining thatthe point y is located below the point x, and determining that an anglebetween gradient vector v₁ and gradient vector v₂ is within apredetermined angle range. According to another aspect of the invention,determining that the point x is located on the air-tissue interfacefurther includes determining that an image intensity at the point x isgreater than a first intensity threshold and less than a secondintensity threshold. In accordance with yet another aspect of theinvention, determining that the point y is located on the air-tissueinterface further includes determining that an image intensity at thepoint y is greater than a first intensity threshold and less than asecond intensity threshold. Determining that the point y is locatedbelow the point x can further include determining that v₁(y−x)>0.Considering at least one point x can further include considering allpoints in the digital image that may lie on a polypoid structure, anddetermining whether each point satisfies a first predetermined set ofconditions. According to a further aspect of the invention, the methodcan include determining that a point x identified as a candidate iswithin a predetermined distance of at least one neighbor point x alsoidentified as a candidate, and merging together those points x eachidentified as a candidate into a single merged candidate. Mergingtogether those points x each identified as a candidate can include, foreach point x identified as a candidate, initializing a merged score asthe candidate score of a current selected candidate point x, and, foreach neighbor point x identified as a candidate, resetting the mergedscore to candidate score for neighbor candidate point x if the candidatescore for neighbor candidate point x is greater than the merged score.According to a further aspect of the invention, the method can include,for each selected candidate point x, counting a number N of neighboringcandidate points x that are within a maximum merge distance of theselected point x and retaining those selected candidate points x forwhom the number N is greater than a predetermined minimum merge count.In accordance with yet another aspect of the invention, the method caninclude defining a plurality of rings about the single merged candidate,the plurality of rings having constant width and increasing radius fromthe single merged point, calculating a number N_(v) of neighbors ywithin a predefined distance range of the point x where the points y areon the air-tissue interface and have a gradient magnitude greater thanthe predetermined value, calculating a number N_(c) of neighbors yhaving an intersection score with the single merged candidate greaterthan a predetermined threshold, accumulating the intersection scores ofall N_(c) points y with the single merged candidate, and projecting acomponent of the gradient vector v₂ at each point y onto a plane normalto the gradient vector v₁ at the single merged candidate, where theprojected component v_(x) is defined asv _(x) =v ₂−({circumflex over (v)} ₁ ·v ₂){circumflex over (v)} ₁.

According to another aspect of the invention, there is provided aprogram storage device readable by a computer, tangibly embodying aprogram of instructions executable by the computer to perform the methodsteps for generating candidates from a digital image.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates an exemplary hemispherical growth attached to atissue wall, according to an embodiment of the invention.

FIG. 2 illustrates the correlation of the image gradient of a growthwith a gradient of a Gaussian, according to an embodiment of theinvention.

FIG. 3 illustrates how a flat growth can produce high response at twodifferent scales, according to an embodiment of the invention.

FIGS. 4( a)-(b) illustrate how the local divergence of the gradientfield and the magnitude of the total surface of the correlation spherethat is divergent compete to produce a strong DGFR response, accordingto an embodiment of the invention.

FIG. 5 illustrates an efficient way of predicting how closely twohalf-lines come to intersecting at a single point, according to anembodiment of the invention.

FIG. 6 is a schematic view of how the neighbor positions can beprojected to the tangent plane at the candidate tip, according to anembodiment of the invention.

FIGS. 7( a)-(b) shows an example of a flat growth that is detected by aneedle CG according to an embodiment of the invention.

FIGS. 8( a)-(f) illustrate exemplary colon structures that yield falsepositive candidates that can be detected by a CG according to anembodiment of the invention.

FIGS. 9( a)-(f) depicts some clusters obtained by a needle CG accordingto an embodiment of the invention for various false positive marks andtrue polyps.

FIG. 10 is a graph that illustrates a performance improvement of aneedle CG according to an embodiment of the invention with respect to aprior art CG.

FIG. 11 is a flowchart of a method for generating candidates andfeatures for growth detection in digital image review such as polypdetection in a digital colonoscopy, according to an embodiment of theinvention.

FIG. 12 is a block diagram of an exemplary computer system forimplementing a method for generating candidates and features for growthdetection in digital image review such as polyp detection in a digitalcolonoscopy, according to an embodiment of the invention.

DETAILED DESCRIPTION OF THE INVENTION

Exemplary embodiments of the invention as described herein generallyinclude systems and methods for generating candidates from a digitalimage. Many of the examples given herein relate to generating candidatesfor polyp detection in digital colonoscopy. Accordingly, while theinvention is susceptible to various modifications and alternative forms,specific embodiments thereof are shown by way of example in the drawingsand will herein be described in detail. It should be understood,however, that there is no intent to limit the invention to theparticular forms disclosed, but on the contrary, the invention is tocover all modifications, equivalents, and alternatives falling withinthe spirit and scope of the invention. In particular, while polypdetection in digital colonoscopy is described throughout, the methodsand systems described herein are well adapted for use in detection ofgrowths at any tissue interface in the body that can be imaged usingknown imaging techniques such as MRI, CT, PET, etc.

As used herein, the term “image” refers to multi-dimensional datacomposed of discrete image elements (e.g., pixels for 2-D images andvoxels for 3-D images). The image may be, for example, a medical imageof a subject collected by computer tomography, magnetic resonanceimaging, ultrasound, or any other medical imaging system known to one ofskill in the art. The image may also be provided from non-medicalcontexts, such as, for example, remote sensing systems, electronmicroscopy, etc. Although an image can be thought of as a function fromR³ to R or R⁷, the methods of the inventions are not limited to suchimages, and can be applied to images of any dimension, e.g., a 2-Dpicture or a 3-D volume. For a 2- or 3-dimensional image, the domain ofthe image is typically a 2- or 3-dimensional rectangular array, whereineach pixel or voxel can be addressed with reference to a set of 2 or 3mutually orthogonal axes. The terms “digital” and “digitized” as usedherein will refer to images or volumes, as appropriate, in a digital ordigitized format acquired via a digital acquisition system or viaconversion from an analog image.

Although DGFR CG had been successfully used for the detection of polypsin the analysis of digital images of the colon, there are come practicalissues that effect performance. Relatively minor issues include a highmemory cost and long computation time compared to the cutting planes CGmethods, but more fundamental issues include the fact that its highsensitivity was coupled with an almost tripled number of marks and thatthe candidate location was given as the center of the polyp, rather thanits tip. No experiment had been performed to test the full set of colonCAD features with a larger set of CG candidates to see if the extrafalse positives (FPs) could be eliminated by the classification stagewithout hurting the extra-sensitivity.

The computational cost can be shown to result from three convolutionsrequired for the vector correlation, which can be replaced with a singlescalar convolution between a Gaussian kernel and the divergence of thenormalized image gradient field. In addition, in the absence of internalthresholds on the gradient magnitude, which are present in DGFR CG, thedivergence of the normalized gradient field corresponds to the meancurvature function. This result yields a much faster algorithm andprovides insight and an alternative way to express what the DGFRresponse is measuring: a weighted average of the divergence of thenormalized gradient field over regions defined by the scale parameter σ.This way of describing the response leads to an important realization:candidates for polyp tips should be locations with a relatively highdivergence of the gradient field as measured between the gradient at thecandidate and the gradient at locations on the colon surface at variousdistances from the candidate. The summation over the region is a way to“measure” this divergence at various distances from the tip, but islimited by issues relating to the tip versus center location of thecandidate.

Candidate Location and Scale Selection

Some structures, such as flat polyps or other flat growths, may have ashape that corresponds more to a half ellipsoid than a hemisphere. Inthis case, correlation with a single scale at a time is not welladapted, since the same candidate will produce high response at varyingscales, making it challenging to merge multiple scales at a consistentlocation. Roughly speaking, the DGFR response will be maximal at variousdistances from the tip depending of the scale, making the response moreblurred and less specific. For example, FIG. 3 illustrates how a flatgrowth such as a flat polyp can produce a high response for the same tipT at two different scales and two different candidate locations C₁ andC₂. Merging these two locations and scales into a single candidate seemsa non-trivial task that is not explicitly addressed by the DGFR CG. Thistask can make the combined response weaker than it should for flatgrowths since no single choice of candidate or scale provides thedesired “full” response.

Low Specificity of the Divergence of a Vector Field

Many structures present a locally divergent gradient field. Themagnitude of the response of DGFR depends on two factors: (1) howdivergent is the field locally; and (2) how much of the total surface ofthe correlation sphere is divergent. These two factors compete with eachother in producing a strong response, such that the final response canbe similar for a flat growth such as a flat polyp (low divergence over arelatively large portion of the sphere) or for the ridge of a fold(strong divergence over a small portion of the sphere surface). FIGS. 4(a)-(b) shows this effect schematically and with examples from the colonsurface (FIG. 4( a)), in which locally convergent gradient fields areshown for normal structures and a flat polyp (FIG. 4( b)).

Low Specificity of the Local Average

Computing a local weighted average of the gradient field divergence isalso challenging. There is no single criterion to determine how thisaverage is to be computed, for example, whether to ignore all gradientvectors that are above the plane tangent to the colon surface at thecandidate tip. This issue is aggravated by the fact that vector field“divergence” is very un-specific; the divergence can measure almostarbitrary changes in the direction of the vectors.

A New Candidate Generator

Coming back to FIGS. 3 and 4( a)-(b) that illustrate the issuesmentioned previously, it can be seen that there is a property thatappears to address almost all of these issues (except for the last one:the low specificity of the local averaging): for growths such as polyps,the half lines formed by the candidate tip and the gradient on one handand by a neighbor location on the tissue surface (e.g., the colonsurface) and its corresponding gradient on the other hand, should(almost) intersect at a single point. For flat growths such as flatpolyps, the half lines will intersect or nearly intersect at differentlocations depending on the neighbor, but they will have a point ofintersection or near intersection, whereas for a fold ridge very fewneighbor locations sustain an intersecting or nearly intersecting halfline with the one defined at the candidate tip, even though the gradientfield has high divergence.

According to an embodiment of the invention, there is an efficient wayof predicting whether or not two half-lines intersect at a single point.With notations referring to FIG. 5, depicting a 3D polypoid structure,the two dotted half-lines l₁, l₂ intersect at a single point if and onlyif, the quantity

${s = {\frac{v_{1} \times \left( {y - x} \right)}{{v_{1} \times \left( {y - x} \right)}} \cdot \frac{v_{1} \times v_{2}}{{v_{1} \times v_{2}}}}},$referred to as the intersection score of x and y, is defined and equalto one, where x and y are points on a candidate surface, and v₁, v₂ arethe image gradients at points x and y, respectively. Notice that s isequal to one independently of the angle α, as long as the half-linesintersect at a single point.

To re-phrase the idea, a solution according to an embodiment of theinvention for locating candidates is: let the candidate be the tiprather than the center of the structure of interest. A solutionaccording to an embodiment of the invention for both scale selection andthe low specificity of the gradient divergence is to let the gradientfield at the various locations around the candidate tip define a set ofhalf-lines that can intersect the half-line defined by the gradient atthe tip, independently of how far the half-lines do intersect it butrather than just having converging projections onto a common plane. Inaddition, a method according to an embodiment of the invention canaddress the low specificity of the smoothing operation by replacing itwith an explicit non-linear algorithm on the neighborhood of eachcandidate tip as follows.

For each voxel x with intensities within a tissue interface such as ablood-tissue interface, or an air-tissue interface and with sufficientlylarge image gradient on the tissue surface:α₁ <I(x)<α₂ ,|v ₁|<α₃  (1)

where v₁ is the gradient of the image I at point x, consider all voxelsy that satisfy the following conditions.

The voxel y is in the air-tissue interface:α₁ <I(y)<α₂  (2.1)The gradient v₂ at point y is sufficiently large:|v ₂|>α₃  (2.2)The distance to the candidate x is within a given range:α₄ <|y−x|<α ₅  (2.3)The neighbor y lies “below” the candidate x:v ₁·(y−x)>0  (2.4)The angle between the two gradient vectors is within a given range:θ_(t) <v ₁ ·v ₂<θ_(u).  (2.5)

A valid neighbor is a neighbor y that satisfies the above conditions 2.1to 2.5. Then, each neighbor location y contributes to a score at x avalue s_(y) given by the intersection score of x and y:

$\begin{matrix}{{s_{y} = {\frac{v_{1} \times \left( {y - x} \right)}{{v_{1} \times \left( {y - x} \right)}} \cdot \frac{v_{1} \times v_{2}}{{v_{1} \times v_{2}}}}},} & (3)\end{matrix}$

whenever s_(y) is above a predefined intersection score threshold θ_(t),(generally close to one, for example, 0.7, 0.8, etc.), and an angle θbetween the two gradients v₁, v₂ is within the predefined rangeθ_(l)<θ<θ_(u). A valid neighbor y whose value of s_(y) is above thepredefined intersection score threshold θ_(t) is said to be asubstantially coplanar neighbor of candidate x. According to anembodiment of the invention, exemplary, non-limiting values for theseconstants for a CT image are as follows: α₁=200 HU, α₂=700 HU, α₃=250HU/mm, θ_(l)=3 degrees, and θ_(u)=55 degrees. Note that these values andunits may differ for images acquired through other imaging modalities,such as magnetic resonance imaging. For all candidate points x, thoses_(y)'s for neighbors that are substantially coplanar with candidate xare summed,

$\begin{matrix}{{S = {\sum\limits_{y}s_{y}}},} & (4)\end{matrix}$

where S is referred to as the candidate score, subject to the optionalfurther condition that at least a fraction φ of valid neighbors of y arealso substantially coplanar with x. φ may be referred to as thecoplanarity fraction. Exemplary, non-limiting values for θ_(t) and φ are0.60 and 0.80, respectively. Finally, only those points x whose value ofS is greater than a minimum candidate score S_(min) are selected ascandidates. An exemplary, non-limiting value for S_(min) is 15.0. Acandidate generator according to an embodiment of the invention for aCAD system that relies on EQ. (3) above to determine if line segmentsextended from gradient vectors intersect or nearly intersect is referredto as a needle CG.

The result could be depicted as clusters of marks, with one mark foreach accepted candidate. Since many of these candidates would refer to asame growth or polyp, the number of candidates is reduced by combining(or merging) candidates that are near another candidate, that is, withina maximum merge distance d₁, keeping only the strongest candidate.According to an embodiment of the invention, an exemplary, non-limitingvalue for d₁ is 10 mm. Thus, after the candidates are selected, for eachselected candidate, the system counts the number of neighboringcandidates whose candidate score S is greater than a minimum thresholdβ₁. According to some embodiments of the invention, minimum threshold β₁is equal to the minimum candidate score S_(min). Further, optionally,according to some embodiments of the invention, the number ofneighboring candidates should be greater than a minimum merge count β₂for the selected candidate to be retained for merging. In otherembodiments of the invention, the minimum merge count β₂ may be set toone.

Next, the selected candidates are visited. For each currently visitedcandidate x, the system initializes a merged score as the candidatescore S for current candidate x and visits the neighboring candidates ythat are within the maximum merge distance d₁ of the current candidatex. However, if one of the neighboring candidates y has candidate scorethat is greater than a current merged score, the merged score is resetto the candidate score S for that candidate y. The effect of the mergingoperation is to find the candidate with the strongest candidate score Swithin a cluster of candidates within the maximum merge distance d₁ ofeach other. After performing the merging, there is presumably onecandidate per potential growth or polyp, referred to herein as themerged candidate or merged candidate point, and the merged candidatepoint is often associated with a tip of the potential growth or polyp.

The above calculations can be represented by the flowchart of FIG. 11.FIG. 11 describes the steps in accordance with the disclosed methodspecifically for detection of polyps in a digital colonoscopy. However,this is merely exemplary, as the disclosed method can be used to detectgrowths in other areas of the body, namely at any visualizable tissuebarrier. For the digital colonoscopy review of FIG. 11, the tissuebarrier is the air-tissue barrier, but for other body locations, thetissue barrier can be the blood-tissue barrier or bone-tissue barrier,etc. Referring now to the figure, given a digital image of the colon, amethod according to an embodiment of the invention begins at step 110 byfinding all points x on the air-tissue interface with a sufficientlylarge gradient, as indicated by the conditions of EQ. (1). Next, at step111, one finds the neighbors y of x that satisfy conditions (2.1) to(2.5). For each such neighbor y, the intersection score s_(y), definedby EQ. (3) is calculated at step 112. At step 113, s_(y) is comparedwith θ_(t) to select those neighbors that are also substantiallycoplanar with x. Then, the s_(y)'s for the substantially coplanarneighbors of x are summed to determine the candidate score S at step 114according to EQ. (4). Steps 111 to 114 are repeated from step 115 forall candidate points x. At step 116, those candidate points for whomS>S_(min) are selected. Then, at step 117, for all selected candidatepoints x, the number of neighbors N for who S>β₁ is determined. Thoseselected points x for whom N<β₂ are eliminated at step 118. At step 119,the remaining points x are merged by finding, within each cluster ofpoints, the point with the highest candidate score S, as describedabove, and selecting that point as the point representative of thecluster of merged points. The merging can be represented by thefollowing pseudo-code.

FOR ALL points x BEGIN x_merge = x S_merge = S(x) FOR ALL points ywithin a distance d of x BEGIN IF S(y) > S_merge THEN S_merge = S(y)x_merge = y END END END

In addition, selected candidates with candidate score larger than athreshold, such as β₁ above, can be taken as candidates for furtherfeature computation. Among the features for selected candidates are thecandidate score S for each selected candidate, the number N_(v) of validneighbors, and the number N_(c) of substantially coplanar neighbors.Among the features of a merged cluster are the number N_(M) ofcandidates actually merged into a cluster, and the sum S_(merge) of allcandidate scores for each candidate merged into a cluster:

$S_{merge} = {\sum\limits_{{merged}\mspace{14mu}{candidates}}{S.}}$According to an embodiment of the invention, the symmetry of a clustermay be determined by considering a plurality of ring-like bands ofincreasing radius centered about the merged candidate point. Accordingto an exemplary, non-limiting embodiment of the invention, there are tenring-like bands of increasing radius, each having a width of 1 mm, aboutthe merged candidate point. Within each band, one counts the number ofvalid neighbors, the number of substantially coplanar neighbors, andsums the candidate scores for all valid neighbors in the band. One canalso project the gradient v₂ of each valid neighbor onto the planenormal to the gradient v₁ associated with the merged candidate point,which is tangent to the tip of the cluster, as shown in FIG. 6. Thegradient v₂ can then be written as v₂=({circumflex over(v)}₁·v₂){circumflex over (v)}₁+v_(x), with v_(x) being the projectedvector, and

${{\hat{v}}_{x} = \frac{v_{2} - {\left( {{\hat{v}}_{1} \cdot v_{2}} \right){\hat{v}}_{1}}}{{v_{2} - {\left( {{\hat{v}}_{1} \cdot v_{2}} \right){\hat{v}}_{1}}}}},$where the hat symbol indicates a vector of unit magnitude. FIG. 6 is aschematic view of how the neighbor positions can be projected to thetangent plane at the candidate tip in order to be able to measureangular symmetry, height and height symmetry among voxels in a cluster.The resulting ring of outwardly projecting vectors is a measure of therelative symmetry of the merged cluster.

FIGS. 7( a)-(b) shows an example of a challenging flat growth 70 (FIG.7( a)) that is detected by a CG (FIG. 7( b)) according to an embodimentof the invention, previously missed using a cutting planes CG. Thearrows 71 in FIG. 7( b) represent the gradients whose of points ofintersections are being sought, and the point 72 is the growth tip.

The above features computed from the selected candidate points andmerged candidates enable identification of certain anatomical structuresthat can yield false positive candidates. For example, FIGS. 8( a)-(f)illustrate colon structures that yield false positive candidates thatcan be detected by a CG according to an embodiment of the invention.Among these structures are fold bending (FIG. 8( a)), a balloon at theend of a rectal tube (FIG. 8( b)), the end of a fold (FIG. 8( c)), twofolds coming together at a junction (FIG. 8( d)), a crumpled fold (FIG.8( e)), and a fold junction that is also bent (FIG. 8( f)). FIGS. 9(a)-(f) depicts some clusters obtained for various false positivecandidates and true positive candidates by a needle CG according to anembodiment of the invention. In particular, FIGS. 9( a) and (b) on thetop row depict false positive structures, FIGS. 9( c) and (d) in themiddle row depict true growths or polyps, and the bottom row of figures,FIGS. 9( e) and (f) depict false positive structures. The darkenedsquares 90 in the structures represent candidate points. For claritypurposes, the squares are only indicated for FIG. 9( a). The relativeasymmetry of false positive structures and symmetry of the true polypsmay be seen from the distribution of candidate points in the figures.

Performance

FIG. 10 is a graph that shows the performance improvement on flatgrowths on a testing set of a baseline CAD system using a prior art CGand a CAD system using a needle CG according to an embodiment of theinvention. The graph plots a volume level sensitivity as a function ofthe number of false positive per volume for the baseline CAD system 101and for a CAD system using a needle CG 103 according to an embodiment ofthe invention. The dotted lines 105 and 106 represent the limitingvalues of plots 101 and 103, respectively, if extended to the right pastthe border of the graph. It may be seen from the plot for the baselineCAD system that it will detect 65.93% of the true growths or polyps in asample of approximately 140 candidates, with an operating point 102 of54% sensitivity at a mean false positive value of 3.5 per volume.Similarly, the plot for a CAD system using a needle CG according to anembodiment of the invention will detect 95.56% of the true growths orpolyps in a sample of approximately 148 candidates, with an operatingpoint 104 of 73% sensitivity at a mean false positive value of 3.17 pervolume.

System Implementations

It is to be understood that embodiments of the present invention can beimplemented in various forms of hardware, software, firmware, specialpurpose processes, or a combination thereof. In one embodiment, thepresent invention can be implemented in software as an applicationprogram tangible embodied on a computer readable program storage device.The application program can be uploaded to, and executed by, a machinecomprising any suitable architecture.

FIG. 12 is a block diagram of an exemplary computer system forimplementing a method for generating candidates from a digital image,such as in digital colonoscopy, according to an embodiment of theinvention. Referring now to FIG. 12, a computer system 121 forimplementing the present invention can comprise, inter alia, a centralprocessing unit (CPU) 122, a memory 123 and an input/output (I/O)interface 124. The computer system 121 is generally coupled through theI/O interface 124 to a display 125 and various input devices 126 such asa mouse and a keyboard. The support circuits can include circuits suchas cache, power supplies, clock circuits, and a communication bus. Thememory 123 can include random access memory (RAM), read only memory(ROM), disk drive, tape drive, etc., or a combinations thereof. Thepresent invention can be implemented as a routine 127 that is stored inmemory 123 and executed by the CPU 122 to process the signal from thesignal source 128. As such, the computer system 121 is a general purposecomputer system that becomes a specific purpose computer system whenexecuting the routine 127 of the present invention.

The computer system 121 also includes an operating system and microinstruction code. The various processes and functions described hereincan either be part of the micro instruction code or part of theapplication program (or combination thereof) which is executed via theoperating system. In addition, various other peripheral devices can beconnected to the computer platform such as an additional data storagedevice and a printing device.

It is to be further understood that, because some of the constituentsystem components and method steps depicted in the accompanying figurescan be implemented in software, the actual connections between thesystems components (or the process steps) may differ depending upon themanner in which the present invention is programmed. Given the teachingsof the present invention provided herein, one of ordinary skill in therelated art will be able to contemplate these and similarimplementations or configurations of the present invention.

While the present invention has been described in detail with referenceto exemplary embodiments, those skilled in the art will appreciate thatvarious modifications and substitutions can be made thereto withoutdeparting from the spirit and scope of the invention as set forth in theappended claims.

What is claimed is:
 1. A method of generating candidates from a digitalimage by a computer system, comprising: acquiring by a computer systemat least one digital image; analyzing, by the computer system, at leastone point x that may lie on a polypoid structure, and determining, bythe computer system, whether the point x satisfies a first predeterminedset of conditions; for each point x that satisfies the predetermined setof conditions, identifying, by the computer system, each neighbor pointy within a predetermined distance of point x that satisfies a secondpredetermined set of conditions; determining, by the computer system, agradient vector v₁ for point x and identifying, by the computer system,a first half-line to which the gradient vector v₁ belongs; determining,by the computer system, a gradient vector v₂ for point y andidentifying, by the computer system, a second half-line to which thegradient vector v₂ belongs; calculating, by the computer system, anintersection score that represents how close the first and secondhalf-lines come to intersecting; and identifying and outputting, by thecomputer system, point x as a candidate when a candidate score isgreater than a predetermined value, wherein the candidate score is thesum of intersection scores for all neighbor points y.
 2. The method ofclaim 1, wherein determining whether a point x satisfies the firstpredetermined set of conditions comprises: determining, by the computersystem, that the point x is located on the air-tissue interface; anddetermining, by the computer system, that the point x has a gradientmagnitude greater than a predetermined gradient value.
 3. The method ofclaim 2, wherein determining, by the computer system, that the point xis located on the air-tissue interface further comprises determining, bythe computer system, that an image intensity at the point x is greaterthan a first intensity threshold and less than a second intensitythreshold.
 4. The method of claim 1, wherein determining whether a pointy satisfies the second predetermined conditions, comprises: determining,by the computer system, that the point y is located on the air-tissueinterface; determining, by the computer system, that the point y has agradient magnitude greater than a predetermined gradient value;determining, by the computer system, that the point y is located withina predetermined distance range of the point x; determining, by thecomputer system, that the point y is located below the point x; anddetermining, by the computer system, that an angle between gradientvector v₁ and gradient vector v₂ is within a predetermined angle range.5. The method of claim 4, wherein determining that the point y islocated on the air-tissue interface further comprises determining thatan image intensity at the point y is greater than a first intensitythreshold and less than a second intensity threshold.
 6. The method ofclaim 4, wherein determining, by the computer system, that the point yis located below the point x further comprises determining thatv₁·(y−x)>0.
 7. The method of claim 1, further comprising: determining,by the computer system, that a point x identified as a candidate iswithin a predetermined distance of at least one neighbor point x alsoidentified as a candidate; and merging, by the computer system, togetherthose points x each identified as a candidate into a single mergedcandidate.
 8. The method of claim 7, wherein merging, by the computersystem, together those points x each identified as a candidatecomprises, for each point x identified as a candidate: initializing, bythe computer system, a merged score as the candidate score of a currentselected candidate point x; and for each neighbor point x identified asa candidate, resetting, by the computer system, the merged score to thecandidate score for the neighbor candidate point x if the candidatescore for the neighbor candidate point x is greater than the mergedscore.
 9. The method of claim 7, further comprising, for each selectedcandidate point x, counting, by the computer system, a number N ofneighboring candidate points x that are within a maximum merge distanceof the selected point x and retaining those selected candidate points xfor whom the number N is greater than a predetermined minimum mergecount.
 10. The method of claim 1, wherein the intersection score isdefined by${{- \frac{v_{1} \times \left( {y - x} \right)}{{v_{1} \times \left( {y - x} \right)}}} \cdot \frac{v_{1} \times v_{2}}{{v_{1} \times v_{2}}}},$wherein v₁ is the gradient vector at point x and v₂ is the gradientvector at point y.
 11. The method of claim 1, wherein analyzing, by thecomputer system, at least one point x further comprises analyzing, bythe computer system, all points in the digital image that may lie on apolypoid structure, and determining, by the computer system, whethereach point satisfies a first predetermined set of conditions.
 12. Aprogram storage device readable by a computer, tangibly embodying aprogram of instructions executable by the computer to perform the methodsteps for generating candidates from a digital image, the methodcomprising the steps, implemented by the computer, of: acquiring atleast one digital image; analyzing at least one point x that may lie ona polypoid structure, and determining whether the point x satisfies afirst predetermined set of conditions; for each point x that satisfiesthe predetermined set of conditions, identifying each neighbor point ywithin a predetermined distance of point x that satisfies a secondpredetermined set of conditions; determining a gradient vector v₁ forpoint x and identifying a first half-line to which the gradient vectorv₁ belongs; determining a gradient vector v₂ for point y and identifyinga second half-line to which the gradient vector v₂ belongs; calculatingan intersection score that represents how close the first and secondhalf-lines come to intersecting; and identifying, and outputting, pointx as a candidate when a candidate score is greater than a predeterminedvalue, wherein the candidate score is the sum of intersection scores forall neighbor points y.
 13. The computer readable program storage deviceof claim 12, wherein determining whether a point x satisfies the firstpredetermined set of conditions comprising: determining that the point xis located on the air-tissue interface; and determining that the point xhas a gradient magnitude greater than a predetermined gradient value.14. The computer readable program storage device of claim 13, whereindetermining that the point x is located on the air-tissue interfacefurther comprises determining that an image intensity at the point x isgreater than a first intensity threshold and less than a secondintensity threshold.
 15. The computer readable program storage device ofclaim 12, wherein determining whether a point y satisfies the secondpredetermined conditions comprises: determining that the point y islocated on the air-tissue interface; determining that the point y has agradient magnitude greater than a predetermined gradient value;determining that the point y is located within a predetermined distancerange of the point x; determining that the point y is located below thepoint x; and determining that an angle between gradient vector v₁ andgradient vector V₂ is within a predetermined angle range.
 16. Thecomputer readable program storage device of claim 15, whereindetermining that the point y is located on the air-tissue interfacefurther comprises determining that an image intensity at the point y isgreater than a first intensity threshold and less than a secondintensity threshold.
 17. The computer readable program storage device ofclaim 15, wherein determining that the point y is located below thepoint x further comprises determining that v₁·(y−x)>0.
 18. The computerreadable program storage device of claim 12, the method furthercomprising: determining that a point x identified as a candidate iswithin a predetermined distance of at least one neighbor point x alsoidentified as a candidate; and merging together those points x eachidentified as a candidate into a single merged candidate.
 19. Thecomputer readable program storage device of claim 18, wherein mergingtogether those points x each identified as a candidate comprises, foreach point x identified as a candidate: initializing a merged score asthe candidate score of a current selected candidate point x; for eachneighbor point x identified as a candidate, resetting the merged scoreto the candidate score for the neighbor candidate point x if thecandidate score for the neighbor candidate point x is greater than themerged score.
 20. The computer readable program storage device of claim18, the method further comprising, for each selected candidate point x,counting a number N of neighboring candidate points x that are within amaximum merge distance of the selected point x and retaining thoseselected candidate points x for whom the number N is greater than apredetermined minimum merge count.
 21. The computer readable programstorage device of claim 12, wherein the intersection score is defined by${{- \frac{v_{1} \times \left( {y - x} \right)}{{v_{1} \times \left( {y - x} \right)}}} \cdot \frac{v_{1} \times v_{2}}{{v_{1} \times v_{2}}}},$wherein v₁ is the gradient vector at point x and v₂ is the gradientvector at point y.
 22. The computer readable program storage device ofclaim 12, wherein acquiring at least one point x further comprisesconsidering all points in the digital image that may lie on a polypoidstructure, and determining whether each point satisfies a firstpredetermined set of conditions.